Advanced Intelligent Systems
○ Wiley
Preprints posted in the last 90 days, ranked by how well they match Advanced Intelligent Systems's content profile, based on 11 papers previously published here. The average preprint has a 0.01% match score for this journal, so anything above that is already an above-average fit.
Palangattu, A.; Sah, A. K.; Raman, S.; Pushpavanam, K. S.
Show abstract
In materials science, the integrity of scanning electron microscopy (SEM) images is paramount for quality control and validation of research outcomes. However, the introduction of sophisticated generative artificial intelligence, particularly Generative Adversarial Networks (GANs), has introduced a novel vulnerability: the potential for highly realistic, artificially synthesized SEM images to be used fraudulently in scientific literature. To address this challenge, we present a deep learning-based framework capable of distinguishing between authentic SEM images and those synthesized by Generative Adversarial Networks (GANs). Using FastGAN and StyleGAN2-ADA, two state-of-the-art GAN models, we generated synthetic SEM datasets to complement real imaging data. We fine-tuned a pre-trained Contrastive Language-Image Pre-training (CLIP) Vision Transformer (ViT-L-14) for binary classification. By unfreezing the final transformer blocks and appending a custom classification head, the model effectively captures the subtle, high-level artifacts inherent in GAN-generated upsampling. This work highlights the potential of deep learning to safeguard scientific imaging workflows and provides an important step toward detecting and mitigating image forgeries in materials science publications.
Hu, Y.; Wang, J.; Zheng, K.; Yu, H.
Show abstract
Virtual staining provides a non-invasive alternative to fluorescence microscopy, yet existing deep learning approaches fundamentally rely on pixel-aligned, multiplexed fluorescence targets for supervision. This dependence on rigidly paired data limits scalability, constrains flexibility in generating diverse subcellular structures, and becomes impractical in data-scarce biological settings. In this work, we introduce a semantic supervision paradigm for virtual staining, demonstrating that domain-knowledge prompts can effectively replace conventional pixel-level supervision. Unlike existing methods constrained by rigidly paired multiplex targets, our framework leverages biological prompts to decouple structural guidance from image translation. This decoupling enables high-fidelity, independent synthesis of multiple subcellular structures using only single-channel data. To ensure high-fidelity generation under weak supervision, we integrate self-supervised representation learning to mitigate data scarcity and incorporate direct preference optimization to suppress structural artifacts. Evaluations on the JUMP benchmark demonstrate that our approach effectively balances flexibility and fidelity, outperforming supervised baselines with a 43.3 % reduction in Average FID and an Average PCC of 0.912, while exhibiting high robustness in channel-deficient scenarios. Furthermore, the model generalizes across four in-house datasets to successfully multiplex six subcellular structures, overcoming the physical constraints of conventional fluorescent staining.
Cecchi, D.; Lasala, A.; Poli, F.; De Remigis, E.; Moccia, S.; Palagi, S.
Show abstract
Fluorescence microscopy is routinely used in the quantitative analysis of Giant Unilamellar Vesicles (GUVs) because it makes GUVs visible and easy to detect. Nonetheless, it requires fluorescence-equipped microscopes and, especially, the addition of tagged molecules that may introduce undesired and unaccounted variations compared to the pristine formulation. Alternatively, phase-contrast microscopy can be used to observe GUVs without labelling, yet the automatic detection of GUVs in phase-contrast images by standard image processing techniques is difficult. To overcome this limitation, we developed a framework for the automated detection and size characterization of label-free GUVs based on YOLOv11 model, a state-of-art lightweight Convolutional Neural Network (CNN) for object detection. Trained on 164 phase-contrast microscope images containing 4666 manually annotated vesicles, the model enables robust detection under low contrast-to-noise ratio and heterogeneous illumination. The trained model is implemented in a stand-alone desktop application supporting multiple analysis in a single run, pixel-to-micrometer calibration, and automated size distribution analysis. This approach achieves high detection accuracy and competitive cross-microscope generalization using YOLOv11 nano on greyscale image. Retraining scripts compatible with standard hardware are also provided to enhance adaptability. This tool enables fast and reliable analysis of GUVs without altering their composition, supporting experimental reproducibility.
Erickson, P.; Hazel, D.; Martinez, R.; Shcherbina, K.; Marquez, S. L.; Ferrante, T.; Johnson, K.; Pimkina, A.; Hazan, H.; Mathews, J.; Sesay, A. M.; Levin, M.
Show abstract
Controlling cell physiology is difficult, not only because of cells complexity, but also their capacity for real-time adaptation to interventions, leading to challenges such as drug resistance and transgene silencing. Accumulating evidence suggests that this adaptivity resembles classical forms of learning defined in behavioral science. However, a lack of appropriate platforms has led to gaps in our understanding of cells capacity for adaptive problem-solving in physiological and transcriptional space. Here, we present a device, the Cell Trainer, capable of performing a wide variety of automated training experiments on non-neural mammalian cells, using timed drug pulses as the stimulus, and a mobile fluorescence microscope to capture images of responses, across replicate cultures. The Cell Trainer can operate in either an open-loop (feedforward) or closed-loop (feedback-controlled) mode, and our image analysis pipeline can report the behaviors of individual cells throughout each experiment and quantify population heterogeneity. We showcase the ability of the Cell Trainer to execute experimental protocols and perform single-cell analyses in both modes. We first demonstrate with a feedforward experiment in which myoblasts are repeatedly pulsed with dimethyl sulfoxide (DMSO) and their discrete calcium responses are analyzed, revealing sensitization-like dynamics. Next, we demonstrate a feedback control scheme wherein the fluorescence of a pH/voltage reporter in kidney cells is maintained below a threshold level with controlled pulses of acid. To accelerate research in the field of cell training, learning, and memory, we are openly sharing the Cell Trainer schematics and software with the research community. This platform provides a flexible tool for studying how cellular physiological states can be shaped by patterned stimulation and feedback control through approaches that work with the native adaptive competencies of cells.
Cai, H.; Tian, C.; Yang, Y.; Xing, Y.; Hong, Z.; Chu, H.; Wang, J.; Ao, Z.; Meyer, J. S.; Friend, J.; Tchieu, J.; Gu, M.; Hyun, I.; Mackie, K.; Liu, L.; Guo, F.
Show abstract
Biomimicry has inspired the evolution of robotics toward greater autonomy, adaptability, and symbiosis with humans and dynamic environments. However, current robotic systems still face major challenges in recapitulating the high-efficiency decision-making capabilities of the human brain under complex and dynamic conditions. Here, we present Brainobot, a biohybrid robotic system that establishes a brain organoid controller as a high-level robotic decision-making layer for closed-loop embodiment. By leveraging brain organoid reservoir computing, Brainobot interacts with dynamic environments by receiving and processing sensory inputs and generating motor actions. As a proof-of-concept demonstration, Brainobot is implemented in a humanoid robotic system to perform real-world tasks, including object grasping and laser chasing. Interestingly, Brainobot exhibits unique features, including cross-task adaptivity, high computing efficiency, and low energy consumption. Thus, our approach may provide insights for advancing robotic embodiment and understanding biological decision-making.
Sriratana, N.; Pongsiripreeda, P.; Uasawaengboon, P.; Asavarojpanich, N.; Jocknoi, L.
Show abstract
Cervical cancer is largely preventable when ab- normal cells are detected early. However, manual Pap-smear screening remains labor-intensive because a specimen may con- tain thousands of cells, borderline abnormalities are subtle, and expert cytology review is not equally accessible. Although artificial intelligence can reduce this workload, conventional classifiers often return a label without directly exposing the evidence used and may exploit staining or image-acquisition shortcuts. To address these limitations, we present CERVEX, a pathology-foundation-model framework with an additive class- evidence readout. Each class score is computed as the spatial mean of its class-specific evidence map plus a learned bias; therefore, the heatmap and numerical score are generated by the same forward calculation. CERVEX also produces a numerical research report containing class probabilities, evidence strength, ranked candidate regions, and segmentation-derived morphology, including N:C ratio and nuclear geometry. Under repeated slide- grouped evaluation on RIVA, with labelled target-cohort cells included during training, CERVEX distinguished abnormal from normal cells with AUROC 0.911 (SD 0.009), sensitivity 0.869, and specificity 0.796. Furthermore, fixed-grid analysis without cell coordinates distinguished low- from high-grade disease across 101 graded abnormal slides with AUROC 0.823 [0.728, 0.911]. On SIPaKMeD, nucleus and complete-cell segmentation reached Dice 0.932 and 0.942, while nuclear area fraction and N:C ratio reached measurement correlations of 0.977 and 0.962, respectively. Moreover, CERVEX produced class-specific spatial evidence while a separate held-out CRIC comparison did not resolve an accuracy difference from the conventional classifier, demonstrating computation-linked explainability with only 1.9% measured runtime overhead. Ultimately, CERVEX enables auto- mated analysis of cervical cytology microscope fields by linking cell classification, computation-linked spatial evidence, and in- terpretable numerical morphology indices in one pipeline. This supports transparent candidate-region review and establishes a practical foundation for future whole-slide screening and prospective clinical validation.
Hobson, C. M.; Puls, O. F.; Aaron, J. S.; Denans, N.; Schmidt, A.; Farrants, H.; Schreiter, E. R.; Chew, T.-L.
Show abstract
The lifetime of fluorescent molecules provides an orthogonal readout to fluorescence intensity, opening experimental possibilities of measuring changes in local molecular environments, mechanical tension, and metabolism, among other factors. These changes are best studied live and in vivo; however, limitations of slow imaging speeds, high phototoxicity, and increased data size and complexity have significantly impeded progress on this front. Here, we present a complete and transferable pipeline consisting of a light sheet FLIM microscope and an accompanying machine learning model for data processing that renders long-term and/or high-speed volumetric FLIM (vFLIM) tractable in living systems. We benchmark this pipeline across several biological use cases, model systems, lifetime ranges, and spatiotemporal scales, showcasing a suite of possibilities that our workflow enables. This comprehensive pipeline from imaging to analysis is a crucial step forward towards disseminating the power of live vFLIM to the broader bioimaging community.
Rossi, I.; Meier, E. K.; Nanes Sarfati, D.; Guadalupe Zamora, F.; Fung, S.; Cleves, P. A.; Herr, A.
Show abstract
The sea anemone Aiptasia is a model system for understanding cnidarian loss of symbiotic algae under heat stress (bleaching). While Aiptasia polyps have been widely used to study this process, accurate symbiosis phenotyping grapples with discordant length scales: fine spatial resolution (~100 um) is needed across a whole organism (~5 mm). To address this, we consider small (~100 um), optically transparent Aiptasia larvae as a bleaching model suitable for whole-organism phenotyping by fluorescence microscopy with larvae classified as symbiotic when algae are localized within gastrodermal cells. To expedite phenotyping, we introduce a machine-learning (ML) image-analysis pipeline (SYMPHONY) designed for single-larva resolution analysis of intact larvae. SYMPHONY efficiently identifies the cellular location of internalized algae (accuracy: 79%, precision: 82%, recall: 79%, F1 score: 79%; training dataset composed of 1611 total objects). Additionally, SYMPHONY reports statistically significant larval bleaching under heat stress and corroborates manual phenotyping results, while significantly reducing operator labor from hours to minutes. The combination of the Aiptasia larvae model and the SYMPHONY pipeline aims to accelerate our understanding of symbiosis breakdown.
Chen, J.; Xu, F.; Jablonski, P. J.; Kuranov, R.; Liu, X.; Hu, Y.; Sun, C.; Zhang, H. F.
Show abstract
Visual neuroscience requires precise spatiotemporal projection of optical stimulation onto the retina, especially in experimental mouse models. However, in vivo patterned stimulation in mice is profoundly hindered by the extreme optical power and severe anatomical aberrations of the eye. Consequently, visual stimulation relies mainly on unverifiable, open-loop approximations that often lack spatial precision. Here, we introduce a closed-loop, spatially modulated stimulation platform that overcomes these barriers. By integrating a digital micromirror device (DMD) with electronically tunable lenses (ETLs) and a real-time, fundus camera-guided focus optimization module, we directly verify the location of patterned stimuli on the retina while dynamically correcting for chromatic and geometric defocus. This platform delivers quantitatively verified static and dynamic patterned stimuli to the living retina with lateral resolutions as fine as 6.7 {micro}m. Guided by ray-tracing optical analysis, our work establishes a technological foundation that enables highly reproducible, cellular-scale interrogations of the visual pathway.
Deng, Q.; Hu, G.; Rauscher, B. C.; Villafuerte, M.; Weinberg, B.; Chen, Z.; Feng, H.; Devor, A.; Thunemann, M.; Tian, L.
Show abstract
Understanding biological function often requires complementary information on molecular identity, morphology, behavior, and physiology acquired simultaneously from the same specimen. Achieving this capability in miniature microscopes remains challenging because integrating multiple imaging contrasts typically increases optical complexity while compromising field of view (FOV), spatial resolution, or form factor. Here we present dual-mode Bio-CM2, a multimodal computational miniature mesoscope that simultaneously captures fluorescence and label-free reflectance through a shared optical architecture based on distributed computational optics. The system enables frame-interleaved, co-registered dual-mode imaging over a [~]7.5 x 10 mm2 FOV with [~]6 m lateral resolution. We demonstrate the versatility of dual-mode Bio-CM2 across diverse biological systems. In freely moving Caenorhabditis elegans, reflectance captures body posture and locomotor behavior that contextualize fluorescently labeled protein aggregates. In freely swimming larval zebrafish, reflectance captures whole-body morphology and swimming behavior, while fluorescence visualizes cardiac activity. In head-fixed mice, simultaneous fluorescence and reflectance imaging across the bilateral dorsal cortex provides complementary measurements of neuronal calcium activity and intrinsic hemodynamic signals. By integrating molecularly specific fluorescence with complementary reflectance contrast, dual-mode Bio-CM2 extends distributed computational optics into a scalable platform for multimodal miniature imaging.
Chen, J.; Chen, W.
Show abstract
Three-dimensional medical image segmentation is unusually heterogeneous in its computational needs: large homogeneous organ interiors are often resolved at coarse scale, whereas small organs, lesions, thin vessels, and ambiguous boundaries benefit from native-resolution processing. Yet modern 3D segmentation networks allocate nearly uniform computation across the entire volume. Existing dynamic-resolution or coarse-to fine methods reduce cost, but typically route regions using hand-crafted image complexity, prediction confidence, or a fixed crop, none of which directly estimates whether spending additional computation on a region will improve the segmentation. We introduce ERRORROUTE, a budgeted inference framework that learns the marginal refinement utility of candidate 3D blocks. A coarse branch first predicts global low-resolution semantics. During training, we estimate for each candidate block the loss reduction achievable by native resolution refinement and supervise a utility estimator to predict this gain before refinement. A differentiable budgeted selector then allocates a fixed compute budget to blocks with the highest expected utility per unit cost, while unselected regions remain on the efficient global path. Selected blocks are refined locally and merged through residual cross-scale fusion. Across six 3D CT/MRI benchmarks, ERRORROUTE achieves a mean Dice of 90.5% at 128 GFLOPs, improving the accuracy-compute frontier over dense 3D baselines and fixed coarse-to-fine systems. At a 30% routing budget, the learned utility estimator recovers 83% of the blocks selected by an oracle router, with the largest allocation concentrated on lesions, small structures, and complex boundaries. These results suggest that efficient medical segmentation benefits from learning where additional computation is useful, rather than merely where the model is uncertain.
Rashed, S. K.; Nilsson, M.; Aits, S.
Show abstract
Convolutional neural networks (CNNs) have shown strong capabilities for image analysis. However, deploying these models in medical settings is complicated by their limited transparency. Over recent years, many approaches have been developed to overcome the so-called "black box" problem of deep neural networks. Here, we show how such explainable AI (XAI) approaches can be applied to not only improve transparency but also training data quality and model performance, with classification of lung damage in histopathology images as use case. First, we conducted a thorough exploratory data analysis and visually compared the compressed multi-dimensional representations of the histology images from the last CNN layers with labels given by pathologists to reveal flaws in the training data. Second, we used Gradient-based Class Activation Mapping (Grad-CAM) as well as SHapley Additive exPlanations (SHAP) values to identify image regions that significantly contributed to the model decisions. To overcome identified shortcomings, we then finetuned additional top layers of the CNNs and introduced model architectures with attention which improved model performance. In summary, we developed a practical workflow that uses interpretability analyses to examine model perception, assess label consistency, and guide model refinement on the example of lung histopathology scoring, demonstrating how XAI approaches can increase both transparency and model performance. The code for this paper is shared at https://github.com/Aitslab/Histology_XAI.git.
Gentsch, G. J.; Guo, M.; Platz, A.; Brehm, G.; Hennings, J. C.; Huebner, C. A.; Stark, A. W.; Franke, C.
Show abstract
Surface phenotyping underpins plant science, preclinical animal research and entomology, yet across all three the measurement is almost always a photograph, which records a projection and not the surface itself. Here we present the Gentschinator3000, an open structured-light platform that brings high-end metric surface measurement within reach of laboratories with no optics expertise, combining documented open hardware, open reconstruction software and analysis workflows for under 4000 Euro in components. It resolves a planar reference to 45 m local flatness, registers full rotations to a loop closure of 156 m, and performs stably across acquisition ranges that we define. Applying one workflow to a leaf before and after desiccation, to murine anatomy and to a spread lepidopteran, we find that projection underestimates surface area by 11 to 41 %. That error grows with the condition under study, with the evaluation scale and with the direction of view, so it can confound phenotype comparisons dramatically. In murine limbs a 15-degree change of viewing direction shifts a projected inter-segment angle by up to 23.2 degrees, while the three-dimensional angle does not move. Projection geometry can therefore contribute as much to a measured phenotype as the biology it is meant to quantify.
Zhao, W.; Wang, N.; Han, Z.; Xu, X.; Ma, X.; Zhou, T.; Gui, J.; Liu, Y.; Liu, Q.; Qu, L.; Liu, W.; Ding, X.; Liu, X.; Ta, D.; Tan, J.; Chen, L.; Li, H.
Show abstract
Capturing fast hemodynamics in deep organs is essential for understanding microvascular regulation of organ responses to physiological demands and pathological stress. However, non-invasive three-dimensional (3D) imaging of these microscale processes remains challenging due to trade-offs between spatial resolution, imaging speed, and penetration depth. Here, we present fluctuation-based high-order super-resolution acoustic microscope (FLAME), a tracking-free 3D ultrasound imaging technology capable of fast microvascular angiography and flow measurement. Using as few as 30 volumes, FLAME improves 3D ultrasound resolution by 8-fold (~50 m) and shortens data acquisition time by over two orders of magnitude, from tens of seconds to tens of milliseconds, compared with conventional tracking-based super-resolution approaches1-6. This high-throughput capability supports high-fidelity imaging of tissue functions under flexible experimental settings. FLAME achieves an unprecedented 3D super-resolution frame rate of ~40 Hz, capturing transient hemodynamic responses to various physiological and pathological stimuli in different mouse organs, from the microvascular level to the whole-body scale. With open-sourced implementation, FLAME provides an accessible platform for real-time, in-depth hemodynamic monitoring across diverse biomedical applications and clinical translation.
Li, W.-s.; Way, G. P.
Show abstract
Pairing label-free microscopy with virtual staining could reduce the cost and experimental burden of fluorescence microscopy, but its impact is conditional on generalizable inference. Most virtual staining studies assess performance using image quality assessment (IQA) metrics developed for natural images, yet how well these metrics translate to microscopy remains unknown. Here, we examined the behavior of seven commonly-used full-reference training objectives and metrics, MAE, PSNR, SSIM, foreground PSNR and SSIM, LPIPS, and DISTS, under controlled image degradation and realistic out-of-distribution virtual staining. We applied graded intensity, textural, and morphological transformations to Cell Painting images spanning 18 cell lines, seeding densities, and fluorescence channels. Channel, cell line identity and seeding density explained substantial metric variation after controlling for degradation magnitude. DISTS and foreground metrics showed more favorable balance between degradation sensitivity and biological invariance, although no metric reported performance independent of biological context. Incrementally degrading images and evaluating concomitant metric degradation further revealed that most metrics used only a small fraction of their nominal numerical ranges and frequently plateaued while image degradation visibly continued. We next trained three popular virtual staining model architectures (UNet, WGAN-GP, UNeXt) on five U2-OS seeding densities separately, and computed metrics on model predictions across 17 unseen cell lines. We observed that architecture and training U2-OS seeding density together explain less than 2% of metric variation. Visual inspection suggested comparable scores across cell lines correspond to qualitatively distinct errors, such as differences in cell morphology and marker intensity. These findings show that conventional IQA metrics do not effectively translate to virtual staining applications. Selection or optimization of virtual staining models against real application such as in label-free high content drug screening should instead be approached in an application-oriented fashion.
Rapilly, Q.; Badoual, A.; LIU, Y.-H.; Kervrann, C.
Show abstract
In this paper, we introduce Nagini3D, a 3D biological image segmentation method that hybridizes two high-performing yet highly complementary segmentation paradigms: deep learning-based approaches and snake-based methods. Our approach leverages both the high efficiency of data-driven techniques and the convenient shape representation provided by active surfaces, which proves highly practical for incorporating a priori knowledge as well as for the post-analysis of segmented objects. By considering a complete 3D volume, our method ensures segmentation consistency across all three dimensions, unlike many approaches that handle volumes in a slice-by-slice manner. Additionally, it theoretically enables the representation of any object with spherical topology, overcoming the limitations of common 3D methods that rely on object convexity assumptions. Furthermore, the continuous representation of objects allows for a resolution-independent description, facilitating the extraction of various geometric features of interest (e.g., local curvature, principal deformations). We demonstrate both the quality of the segmentation produced by our method and the richness of the accessible geometric features through an extensive set of experiments conducted on multiple real and simulated datasets, including objects with challenging geometries.
Vasinkova, M.; Jochymek, L.; Gajdos, P.
Show abstract
Machine learning benchmarks often treat image files as independent observations, although multiple files may represent repeated views or other related samples. Prior work has shown that non-independent partitioning can inflate model performance, but the sensitivity of a biomedical benchmark to progressively increasing train-test relatedness is less often quantified. Here, we quantified how apparent classification performance changes when recoverable sample groups are progressively allowed to cross training, validation, and test partitions. Using source structure and filenames, we reconstructed analysis groups in a microscopy corpus compiled from multiple sources and generated five class stratified partitioning conditions, ranging from an intact group reference to random assignment of individual files. We applied the same training recipe within each model family to 129,173 images spanning 14 classes of blood cells and hematopoietic cell states. For a pretrained Vision Transformer, random assignment of individual files increased the mean test macro F1 from 0.853 to 0.977 and accuracy from 0.851 to 0.990 relative to the intact group reference. Apparent performance increased progressively across the intermediate conditions. The endpoint pattern also appeared with a ResNet-50 backbone, and the largest apparent gains occurred in immature granulocytic and blast classes. A descriptive analysis within the random assignment condition further showed higher scores among test images that had another member of the same group in the training set, although the exposed and unexposed subsets were not exchangeable. The grouping workflow, exact split definitions, code, and compact result files are released as HSD-Bench to support reproducible evaluation of hidden sample dependence.
Alizada, S.; Marks, K. A.; Zitnay, R. G.; Done, A.; Judson-Torres, R. L.; Zangle, T. A.
Show abstract
Cell morphology reflects cell health and can distinguish cell-cycle stage, growth arrest, and distinct pathways of cell death. Live, label-free quantitative phase imaging (QPI) captures these features non-invasively and with high temporal resolution, yet many image-based classifiers rely on single frames and cannot separate states whose differences emerge only over time. How much temporal information is needed, and which architecture best exploits it, remain open questions. We assembled 1,874 QPI timelapse sequences spanning six cell states (interphase, mitosis, cell cycle arrest, apoptosis, ferroptosis, and necroptosis) and compared two-dimensional convolutional neural networks (CNNs) with a three-dimensional (3D) spatiotemporal CNN across increasing frame counts. Accuracy improved as frames were added, with the largest gain between one and three frames. The 2D models saturated beyond three frames, whereas the 3D architecture kept improving, reaching 96.5% accuracy and a 3.5% error rate at eleven frames. The temporal information needed tracked the timescale of each process: mitosis was resolved from a single frame, while ferroptosis benefited most from extended sequences. Overall, these results show that dynamic information, rather than static morphology alone, drives accurate cell-state classification, and that 3D architectures are needed to fully exploit it for label-free dynamic phenotyping.
Vizcarra, J. C.; Kataria, B. C.; Chuah, C.-N.; Pearce, T. M.; Dugger, B. N.; Candano, C. A.; Rosado, A. M.; Cooper, L. A.; Gutman, D. A.
Show abstract
Foundation models (FMs) have rapidly become dominant in artificial intelligence and are increasingly being adopted in computational pathology. Numerous pathology-specific FMs have been developed and evaluated for a variety of downstream tasks, most commonly using frozen image embeddings with linear probes. While a pathology FM, at least implicitly suggests broad reusability across tasks, the transferability of these models to neurodegenerative disease-related tasks remains largely unexplored. In this study, we evaluated fourteen frozen feature extractors, including general-vision and pathology FMs, across four pathology image classification datasets spanning two specialized neurodegenerative disease immuno-histochemistry (IHC) tasks (Tau neurofibrillary tangle (NFT) and amyloid-{beta} plaque classification) and two cancer-related hematoxylin and eosin (H&E) tasks (the breast tissue BACH dataset and the multi-tissue TIL dataset). The cancer-related H&E datasets represent tasks more closely aligned with the predominant pretraining domain of many current pathology FMs, whereas the neurodegenerative IHC datasets represent a more specialized domain with limited representation in existing FM pretraining cohorts. Frozen embedding linear probes were compared against a conventional supervised ResNet-50 convolutional neural network (CNN) trained directly on image tiles using identical train, validation, and test splits. Across both neurodegenerative IHC datasets, the supervised CNN substantially outperformed all frozen FM linear probes. In contrast, pathology FMs achieved performance comparable to, and in some cases exceeding, the supervised CNN across the two cancer-related H&E datasets, with UNI2-h achieving the highest performance on BACH and several pathology FMs performing on par with the CNN on the larger TIL dataset. Furthermore, a supervised CNN trained using only 1% of the Tau NFT training data (1,938 tiles) still exceeded the performance of the best frozen FM linear probe trained on the complete dataset. Together, these results suggest the transferability of frozen pathology FMs may depend strongly on how well the downstream task is represented by their pretraining domain. These findings demonstrate frozen pathology FMs transfer effectively to the cancer-related H&E tasks evaluated here but may be less effective for specialized neurodegenerative IHC tasks less well represented in current FM pretraining cohorts. Expanding pathology FM pretraining datasets to include a broader range of disease domains and staining modalities may therefore improve transferability to specialized pathology applications. Our findings also highlight the continued importance of conventional supervised learning and motivate future work investigating nonlinear probes and end-to-end FM fine-tuning.
Pelloux, M.; Tran, C. T.; Baig, J.; Khellaf, A.; Huang, S. L.; Nguyen, O. T.; Batardiere, M.; Diwan, S.-S.; Yang, Z.; Li, J.-L.; Vu, V.; Roy, S. F.; Soucy, G.; Nguyen, B. N.; Tessier-Cloutier, B.; Hosseini, M. S.; Knapp, D. J.; Valmary-Degano, S.; Trinh, V. Q.
Show abstract
Current state of the art computational pathology foundation models attain satisfying accuracy most general pathology tasks including detection of pancreatic ductal adenocarcinoma (PDAC) but fail to provide sufficient explainability, operating as "black boxes". Some interpretability effort has been made, with models providing attention maps that project back on the slide the models high attended areas that pushed for the given prediction result, without further naming or reasoning explanation which can limit pathologists trust toward the prediction. We developed C3PRO, a cell-centric framework attempting to provide deeper understanding and explainability, by using embeddings extracted using a pathology foundation model, from cell centered tiles containing both the cell and its close neighborhood, which were then clustered into recurrent morphological phenotypes. A spatial graph was additionally constructed on top of the back-projected vocabulary to account for recurrent motifs. The evaluation of the model performance involved a cohort of 809 patients (825 diagnostic biopsies) using nested cross validation. C3PRO was benchmarked against four different multiple instance learning frameworks and achieved on-par performance for diagnostic (0.958 AUC), survival (0.617) and resectability (0.696) prediction, while providing additional insights compared to the strongest baseline models. Each clustered phenotype was probed and extracted as a representative tile mosaic and annotated by two expert pathologists, constituting a phenotype vocabulary available for statistical analysis. These phenotypes were scored by pathologists with physical brightfield microscopy and replicated the independent statistical significance impact on overall survival prediction from diagnostic biopsies.